The set of characters allowed in a Mathematica identifier, of which a reserved word is a special case: «Definition of interesting tokens»= res_word_chars := &letters # ``$'' is also allowed, but we don't include it for now. id_chars := res_word_chars ++ &digits
@ Unbalanced and balanced comment tokens, and quoting tokens. Note that [[comment2]] is a set of pairs (``open comment'', ``close comment''), while we assume that quoting tokens are s.t. the ``open quote'' and ``close quote'' tokens are identical. «Definition of interesting tokens»= comment1 := set([]) comment2 := set([["(*", "*)"]]) quote := set([" ""])
@ The ``special'' tokens. [[S]] is the set of all characters appearing in strings in
[[special]].
«Definition of interesting tokens»=
special := set(["", "", "
", "||", "&&", "<", ">", "==", ">=", "<="])
S := ''
every S := S ++ !special # Nice!
@ The rest of the code is language-independent, and is in the file lipp.nw (Language-Independent Pretty-Printing).